Prioritizing the Propagation of Identity Beliefs for Multi-object Tracking

Multi-object tracking requires locating the targets as well as labelling their identities. Inferring identities of the targets is a challenge when the availability and the reliability of the appearance features do vary along the time and the space. We see the multi-object tracking and identification as a two-stage process. In the first stage, plausible target candidates are detected at each frame independently, and are aggregated into tracklets. The benefits obtained from such aggregation process are twofold. First, it reduces the number of entities that have to be processed later. Second, it provides more reliable and more accurate knowledge about the appearance of the target observed along the tracklet. In the second stage, which embeds the main contributions of the paper, a graph-based belief propagation formalism is considered to estimate the identity of each tracklet. Each node in the graph corresponds to a tracklet, and is assigned a probability distribution of identities, based on the tracklet appearance, and given prior knowledge of the possible target appearances. Typically, a low confidence in the tracklet appearance measurement, or a measurement that is similar to several target appearances, both result into a flat and thus ambiguous identity distribution for the tracklet. Afterwards, belief propagation is considered to infer the identities of more ambiguous nodes from those of less ambiguous nodes, by exploiting the graph constraints. In contrast to the approaches with standard belief propagation [2], which treats the nodes in an arbitrary order, the proposed method schedules less ambiguous nodes to transmit their messages first. From appearance features to identity distribution We assume that there are N targets, each of them being characterized by K appearance features. The feature set for the j-th target is F ( j) = {f j) 1 , ..., f ( j) K }. Let the appearance features for a tracklet v be F (v) = {f 1 , ..., f (v) K }. Then, the probability of the tracklet v having identity j, denoted by pv( j), as pv( j) ∝ K ∏ i=1 exp [ −‖ f j) i − f (v) i ‖1/τ (v) i ] for 1≤ j ≤ N (1) where τ i monitors the influence of feature i on identity assignment. It decreases as the appearance feature observation becomes more reliable. Depending on the observed appearance features and on the estimated reliability of these observations, some tracklets have less ambiguous identity distributions than others. Graph definition The tracklets are gathered into a graph, G = (V,E), where V is a set of nodes, with each node corresponding to a tracklet; E is a set of edges, defining the connectivity between the nodes in V . An edge between nodes u and v implies that their identities are dependent. For example, two tracklets, which co-exist at the same time, should belong to two different physical targets. This defines a mutex edge between them. Additionally, if they are sufficiently close in space, time and/or appearance, they are likely to share the same identity, whereas if they are far, they should be encouraged to have different labels. This defines a temporal edge between them. Each node v ∈ V and each edge (uv) ∈ E is characterized by potential functions φv and φuv respectively. In short, φv(lv) represents how likely is the label lv be assigned to the node v. Similarly, φuv(lu, lv) represents the likelihood that nodes u and v have labels lu and lv respectively. Belief propagation We briefly introduce how the belief propagation formalism works. A graph G = (V,E) is given, where V is the set of nodes and E represents the association between the nodes. The neighbourhood of node v ∈ V is denoted by Nv. The purpose of belief propagation is to find a labelling function l that labels each node v ∈ V with a label lv ∈ L, L being the set of possible labels, so as to maximize the joint likelihood function: p(l) ∝ ∏ v∈V [ φv(lv) ∏ u∈Nv φuv(lu, lv) ] (2)